This translation is not too bad: An analysis of post-editor choices in a machine translation post-editing task

Variation between post-editors of machine translation is a well-known issue. This variation shows itself in post-editing speed, amount of editing and differing final translations. However, relatively few studies exploring the differences have been reported. This paper describes a postediting task involving controlled language tourist phrases translated from English into Finnish. Post-editors select the best out of three machine translated suggestions, which they can accept without editing or post-edit as necessary. Agreement between editors is analyzed and reported in terms of selecting the best suggestion, deciding its acceptability, and producing a final post-edited version. Editors are compared in terms of post-editing time, edit distance and final translations created. With a qualitative analysis, we examine differences between the selected and rejected suggestions as well as differences between the post-edited versions created by different editors. Examples of editor preferences are also discussed.